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1. Introduction 

In this report, the remote sensing system simulation is used to study a 
proposed sensor concept. An overview of the instrument and its parameters is 
presented, along with the model of the instrument as implemented in the 
simulation. Signal-to-noise levels of the instrument under a variety of system 
configurations are presented and discussed. Classification performance under 
these varying configurations is also shown, along with relationships between 
signal-to-noise ratios, feature selection, and classification performance. 

2. Instrument Description 

Driven by the recent advances in optical detector array technology and 
the opportunities for a permanent polar orbiting platform provided by the 
upcoming space station program, research and deyeiopment has been 
progressing on a High Resolution Imaging Spectrometer (HIRIS) [1]. This 
instrument is planned to be a part of a cluster of^ientrfic in^ruments fo g 
the Earth Observing System (EOS) to be launched in the mid 1990 s Th s 
international group of instruments will provide many new forms of scientific data 
of the Earth's surface & atmosphere. 

The HIRIS instrument will provide high spectral resolution samples 
across the visible and near infrared portions of the optical spectrum by the use 
of larqe CCD infrared detector arrays. The incoming radiance will be spectrally 
dispersed onto the arrays to provide one line of spatial data in each of the 
spectral bands simultaneously, as shown in Figure 1. The two dimensional 
image is then formed by the forward motion of the instrument platform. 

In the proposed instrument there will be two separate detector arrays to 
cover the entire spectral range. The Very Near InfraRed (VNIR) array will 
provide 64 bands between 0.4 and 1.0 pm, while the Short Wave InfraRed 
(SWIR) array will contain 128 bands from 1.0 to 2.5 pm. 

Functional parameters of the instrument are summarized in Table > 1. One 
oarticular item of note is that the maximum output data rate is smaller than the 
internal data rate, thereby necessitating data editing or compression be 
performed on board the instrument. 
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Design Altitude 
Ground IFOV 
Swath Width 
Spectral Coverage 

Average Spectral Sample Interval 
0.4- 1.0 pm 
1.0 -2.5 pm 

Pointing 

Down-track 
Cross-track 
Data Encoding 
Maximum Internal Data Rate 
Maximum Output Data Rate 
Image Motion Compensation Gain 


705 Km 
30 m 
20 Km 
0.4 - 2.5 pm 
192 Bands 

9.4 nm 
11.7 nm 

+60%30° 

+ 20 °/- 20 ° 

12 bits/pixel 
512 MBPS 
300 MBPS 
1 . 2, 4, or 8 


Table 1. HIRIS Functional Parameters 
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This instrument will be operated in an on-demand mode by gathering 
data only at the request of a scientific investigator. As part of that request, the 
investigator will need to specify several mission parameters. A typical list is 
given in Table 2. The goal of this report is to investigate sensor performance 
under a variety of mission parameter settings. 


Location of Observation Site on Earth 
Time of Day/Year or 

Zenith Angle of Sun 

Relative Azimuth between Sun and HIRIS 
Look Angle of HIRIS 
Acceptable Atmospheric Quality 
IMC Gain State 
Radiometric Resolution 
Spatial Data Editing/Compression 
Spectral Data Editing/Compression 


Table 2. Example HIRIS Mission Parameters 

3. Instrument Model 

For this investigation the model shown in Figure 2 is used for the HIRIS 
instrument. This model version has 201 equally spaced (10 nm intervals) 
bands from 0.4 to 2.4 pm and includes most major spectral, spatial, and 
radiometric effects of the instrument. The model was implemented in the system 
simulation program described in reference [2]. 

Instrument parameters have been obtained from a progress report by 
jPL (reference [3]). These parameter levels are based upon preliminary 
specifications and prototype testing. 

The following paragraphs and figures detail the blocks in the overall 
diagram. 

31 Optics. The spectral transmissivity of the instrument optics is shown 
in Figure 3. Note the low response at the spectral gap between the VNIR and 

the SWIR arrays at 1 .0 pm. 

The normalized spatial response of the optics and field stop is assumed 
to be similar to the that of the Landsat Thematic Mapper instrument, as they both 
have a G FOV of 30 meters. Figure 4 shows the measured down scene and 
across scene normalized responses as a function of angular distance taken 
from reference [4] The data points shown are the discrete values used in the 
simulation. At the nominal altitude of the instrument, the distance on the ground 
between these data points is approximately 7 meters. 
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Figure 2. HIRIS Model Block Diagram 


4 


4 / 20/89 






0.4 0.6 0.8 1.0 1.2 1-4 1.6 1 -® 2 0 2 2 24 

Wavelength (micrometers) 

Figure 3. Spectral Transmissivity of Optics 



Across Scene 
Down Scene 


Angular Distance (mlcroradtans) 

Figure 4. Normalized Spatial Response 
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The radiometric conversion trom the incoming spectral radiance 
(mW/cm 2 -sr-pm) is accomplished by dividing by 1000 mW/W, and multiplying by 
the AQ the product of the detector area and the solid angle of view of the 
optics 1 The output of the optics model P x , the incident spectral power, is then in 

units of watts/pm. The A£2 used in the model is 

Aft = 1.44 x 10* 6 cm 2 -sr. 

3 p Detector. The spectral quantum efficiency of the detectors is shown 
in Figure 5. The incident spectral power P x at wa v e!ength ^ isconverted to a 
number of electrons S at the detector by the integration of the incident photon 
level over the pixel integration time, as in equation (1). 

S = p x .AX.-j^-t (1) 

AX = 10 nm, wavelength interval of spectral samples 

X = wavelength of interest (pm) 
h = 6.62 x 10- 34 Joule-sec, Planck’s constant 
c = 3 x 10 8 meters/sec, the speed of light 
t = 4.5 millisec, pixel integration time 

Since the noise level data and full scale specifications were obtained in 
terms of number of electrons, the signal level is stated in these same terms and 

is unitless. 

The Image Motion Compensation (IMC) is imp!© m® nted 
movement of the down-track pointing mirror to offset the | p ^°"T d s ^ e ® d r 8 
effectively multiply the pixel integration time by the gain state selected. 2, 4, o . 


where, 
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Wavelength (micrometers) 

Figure 5. Spectral Quantum Efficiency 

^ Noise The noise in this model consists of a deterministic dark sig nal 

s,sH".r xsx jrxs ssssmk 

Sm et its s»r— 

Thus, this type of error has not been included as of yet. 

The dark current level is given in Table 3. This level is added to the total 
received signal. 


VNIR 0 e- 

SWIR 2.7 x 10 4 e- 


Table 3. Dark current levels in terms of electrons 

The shot noise in the model consists of zero mean Gaussian random 
numbers with a standard deviation equal to a function of the tot ^.' ® l ^ a ' '®Y e ' 
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600 



Signal Level (electrons) 

Figure 6. Shot Noise vs. Signal Level 

Thus, the standard deviation of the shot noise process is given by 
equation (2). 

^shot = *^ 060 + 278 log io S ' (2) 

S' is the total detector signal level in electrons. Note, this relationship is 
assumed to be independent of wavelength. 

Read noise is added in as a zero mean Gaussian random number with a 
standard deviation as in Table 4. Within each detector array, the read noise 
level is assumed to be constant over wavelength. 



Table 4. Read Noise Levels 


s 4 Signal r,nnditionina. Table 5 gives the full scale e- values F for each 
detector array. These are values obtained from the JPL report, and as with all of 
the data values presented, should be considered to be subject to change. 
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VNIR 

577,395 e- 

SWIR 

1 .441 ,440 e- 


Table 5. Full Scale Electron Levels F 

The conversion from the e* levels S" (received signal plus noise) to a 
digital number (DN) occurs as in equation (3). 




where, 


DN = nintj-j^Q 


IMC = IMC Gain State 

F = Full Scale Electron Level 

Q = Number of radiometric bits (nominally 1 2) 


(3) 


The division by the IMC gain state is included to preserve the dynamic 
range of the detectors over the various gain states. 

4. Signal to Noise Variations 

This section presents the results of the effect of varying system 
parameters on the signal-to-noise ratio (SNR) of the HIRIS model. 

In many Earth resource analysis remote sensing applications, the output 
product is some form of classification map of the observed area. The 
classification is usually obtained by a computer algorithm that uses the mean 

and covariances of the multispectral image data to Vs 

classes. Thus, in this application, not only are signal levels important, but so 

are signal power variations. 

To gain a more realistic sense of how the system parameters affect HIRIS 
performance, two versions of SNR are defined: Voltage SNR, and Power SN . 
These are defined for each spectral wavelength band m as in equations (4) and 

(5). 


Voltage SNR m = 20 log^ 


Pm 

[f +C 2 „ +0 2 ~ 

V shot.m read.m quant 


(4) 


Power SNR m = 10 log l0 ” 


<4 G m 


o 2 +0 2 +0 2 . . 

shot.m read.m quant J 


(5) 


where, 


P m = Mean surface reflectance at wavelength band m 
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<4 = Variance of surface reflectance at wavelength band m 
G = Conversion factor with units of number of electrons due to the 

m solar irradiance at the surface, atmospheric transmittance, and 
the sensor response for wavelength band m 

°5hot,m - Variance of shot noise in wavelength band m 

°read.m = Variance of read noise in wavelength band m 

°quant = Variance of quantization noise 

The quantization noise is assumed to be uniform with a standard 
deviation of 

- /Z 

°quant \J f2 

where A is the number of electrons per digitization interval. 

For 12 bit radiometric resolution and the full scale values in Table 5, the 
standard deviation of the quantization error is given in Table 6. 

VNIR 41 e- I 

SWIR 102 e- I 


Table 6. Standard Deviations of Quantization Error 


For the SNR results included in this report, the system configuration 
shown in Table 7 was used as a baseline. The ^' a JJ ,,umma ) ,on ' 
atmospheric effects were obtained using the LOWTRAN 7 [5] computer code. 
The 1976 U.S. Standard Atmosphere with rural extinction was used. 


Meteorological Range 

16 Km 

Solar Zenith Angle 

30° 

Solar Azimuth Angle 

180° 

View Zenith Angle 

0° 

View Azimuth Angle 

0° 

IMC Gain State 

1 

Sensor Noise Levels 

Nominal 

Radiometric Resolution 

12 bits 


Table 7. Baseline System Configuration for SNR Study 


Figure 7 shows the voltage SNR for three surface albedos and the 
baseline system configuration of Table 7. 
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LOWTRAN 7 



Albedo - 100 
Afeedo -0.10 
Albedo - 0.01 


Figure 7. SNR of HIRIS for various albedos using LOWTRAN 7 and the 
baseline system configuration given in Table 7. 


For the study of parameter effects on SNR, it was decided to use typical 
surface reflectance statistics rather than a deterministic albedo. The surface 
reflectance statistics used were obtained from a site in Finney County, KS on 
May 3, 1977 [6]. 1551 observations were combined from three classes: winter 
wheat, summer fallow, and an unknown class. The mean reflectance and 
variation of this data ensemble are plotted in Figure 8. 

To obtain an idea of how this reflectance is modified by the atmosphere 
and sensor response, a simulated image was created using the baseline 
system configuration and the reflectance of Figure 8. The resulting mean digital 
counts and their variation are shown in Figure 9. Several effects are 
immediately noticeable. The absorption bands of the atmosphere are present, 
as well as a reversal in the relative values of the visible and infrared responses 
due to the solar illumination and gain settings of the sensor. Also, an apparent 
reduction in relative variation is seen due to the additive constant path radiance 
and dark current in the instrument. No correction for these effects was made. 
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Digital Counts §<£ Reflectance (%) 



'0 8. Mean and variation of the surface reflectance of the combined 
set. 



Figure 9. Mean and variation of image vector as received by HIRIS. 
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The following Figures 10 through 21 show the Voltage and Power SNR 
variations as a function of the parameters shown in Table 8. 


Meteorological Range 
Solar Zenith Angle 
View Zenith Angle 
IMC Gain State 
Sensor Noise Levels 
Radiometric Resolution 


2,4,8,16,32 Km 
0°, 15°, 30°, 45°, 60° 
0°, 15°, 30°, 45°, 60° 
1,2. 4,8 
1/4, 1/2, 1,2,4 
8. 10. 12, 14, 16 bits 


Table 8. Parameters Studied and Their Variations 
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Power SNR (dB) Voltage SNR (dB) 


5° -1 
40 - 
30 - 
20 - 
10 - 
OH 

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 

Wavelength 

Figure 10. Voltage SNR for varying Meteorological ranges. 



32 Km 
16 Km 
8 Km 
4 Km 
2 Km 


0.4 0.6 0.8 1.0 1.2 1.4 1.6 1 


Wavelength 

Figure 11. Power SNR for varying Meteorological ranges. 


14 


4/20/89 




Power SNR (dB) Voltage SNR (dB) 



0 Degrees 
15 Degrees 
30 Degrees 
45 Degrees 
60 Degrees 



0 Degrees 
15 Degrees 
30 Degrees 
45 Degrees 
60 Degrees 


Wavelength 

Figure 13. Power SNR for varying solar zenith angles. 
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Power SNR (dB) Voltage SNR (dB) 


50 



0 Degrees 
15 Degrees 
30 Degrees 
45 Degrees 
60 Degrees 


Wavelength 

Figure 14. Voltage SNR for varying view zenith angles. 



0 Degrees 
15 Degrees 
30 Degrees 
45 Degrees 
60 Degrees 


0.6 0.8 


1.8 2.0 2.2 2.4 


Wavelength 

Figure 15. Power SNR for varying view zenith angles. 
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Power SNR (dB) Voltage SNR (dB) 


70 


60 - 
50 - 
40 - 
30 - 
20 " 

10 - 
0 - 

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 

Wavelength 

Figure 16. Voltage SNR for various IMC gain states. 



60 
50 
40 
30 
20 
10 
0 

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 

Wavelength 

Figure 17. Power SNR for various IMC gain states. 



IMC -8 
IMC- 4 
IMC -2 
IMC-1 


IMC -8 
IMC -4 
IMC - 2 
IMC-1 
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Power SNR (dB) Voltage SNR (dB) 




Wavelength 


Figure 18. Voltage SNR for varying factors 


noise. 



1/4 

1/2 

1 

2 

4 


Wavelength 

Figure 19. Power SNR for varying factors of shot and read noise. 
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Power SNR (dB) Voltage SNR (dB) 


50 




Wavelength 

Figure 21 . Power SNR for various numbers of bits. 


16 bits 
14 bits 
12 bits 
10 bits 
6 bits 


16 bits 
14 bits 
12 bits 
10 bits 
8 bits 
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Several common features of all the graphs are immediately noticeable. 
The wide water absorption bands are present at 1.4 and 1.9 pm, while several 
narrow absorption bands due to other atmospheric constituents are also 
present The relatively low SNR's around 1.0 pm are due to the instrument 
optics and detector response fall off at the spectral gap between the VNIR and 
the SWIR arrays. 

Looking at the effect of the meteorological range, one observes a more 
pronounced effect on the SNR in the visible and near infrared portions of the 
spectrum, than those at longer wavelengths. 

Considering the goniometric configurations, the effects on SNR of 
varying the solar or view angle seem to be similar, at least for the system 
configuration chosen. Note that these variations here are only due to the 
atmospheric effect, as angle dependent reflectance variations of the surface 
were not considered in this model. 

The effect of the IMC and noise level parameters seem to be reasonable 
and consistent. The radiometric resolution has little effect on SNR at resolutions 
of 12 bits or more. This seems to indicate that the quantization error at 12 or 
more bits is insignificant compared to other noise sources, while at 8 or 10 bits 
quantization error becomes the dominant noise source. 
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5. Parameter Effects on Classification Accuracy 


The effects of the parameters in Table 8 were also studied on the 
classification performance of the HIRIS instrument. Simulated scenes were 
generated using the three classes of the Finney County data set of May 3, 1 977 
and the system simulation program [2]. Since the model HIRIS sensor 
generates images with 201 bands, a feature selection algorithm was applied 
that combines bands together to form each feature. For the following 
experiments, a set of 16 features were used with a maximum likelihood 
multivariate classifier (Gaussian assumption) to assess accuracy. Table 9 
shows the wavelength bands combined for each of the features. These features 
are based upon the algorithm described in reference [7]. Table 10 contains 
relevant system parameters used in the simulation. 


Feature 

Wavelength (pm) 

1 

0.70 -0.92 

2 

1.98-2.20 

3 

2.20 - 2.40 

4 

0.66 - 0.84 

5 

1.48-1.64 

6 

0.52 - 0.66 

7 

1.64 -1.78 

8 

1.16-1.28 

g 

0.96-1.06 

10 

1.04-1.12 

ii 

0.94-1.00 

12 

0.44 - 0.50 

13 

1.12-1.16 

14 

0.92 - 0.96 

15 

0.40 - 0.44 

16 

1.00-1.04 


Table 9. Features used in classification experiments. 


Scene Size 90 by 90 Pixels 

Scene Pixel Size 15 meters 

Image Size 45 by 45 Pixels 

Number of Pixels per class 675 

Number of training samples per class 300 


Table 10. System Simulation Parameters. 

The following Figures 22 through 27 show the results of the various 
parameter changes on classification accuracy. The accuracies shown are the 
average of the three class accuracies, and represent the mean of 10 repetitions 
of the simulator. 


21 


4 / 20/09 







Meteorological Range (Km) 

Figure 22. Classification accuracy vs. meteorological range. 



Figure 23. Classification accuracy vs. solar zenith angle. 
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Classification Accuracy (%) 


View Zenith Angle 

Figure 24. Classification accuracy vs. view zenith angle. 
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Figure 25. Classification Accuracy vs. IMC gain state. 
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Figure 26. Classification Accuracy vs. shot and read noise level. 



Number of Bits 

Figure 27. Classification Accuracy vs. number of bits. 
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Most of these results are as one would expect. In hazier atmospheres, or 
with high noise levels the accuracy decreases. Figure 24 showing the effect of 
view angle is a little less intuitively satisfying. The sudden rise in classification 
accuracy at a view angle of 60° is surprising. This seemingly abnormal result is 
due to the complex interaction of the scene pixel size and the sensor spatial 
response. At this view angle the distance on the ground between samples of 
the discrete spatial response becomes approximately 15 meters, the same as 
the size of the scene pixels. At angles less than 60°, about 25 scene pixels 
were appropriately weighted to form one image pixel. Now, at 60°, about 45 
scene pixels are combined. This has the effect of reducing the within class 
variation and increasing the class separability and classification accuracy. 

This result points out how complex interactions between components of 
the system can affect classification accuracy. It also points out some of the 
pitfalls in the use of a discrete simulation for the study of system effects. One 
must exercise caution in specifying system parameters and in interpretation of 
results. 


One significant part of the real instrument not included in the model of 
Figure 2 is error in radiometric calibration. This omission is mainly due to the 
fact that adequate data does not exist to develop a model for this error. To test 
the significance of this omission, a model of the error based on percentage of 
value, and distributed uniformly with a mean of zero, was inserted before the 
signal conditioning. This error is distributed randomly across the image and 
represents a relative error. Figure 28 shows the effect of this error as it varies 
over the range of 0.0 to 4.0 percent. 
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Relative Calibration Error (%) 

Figure 28. Accuracy vs. Relative Calibration Error 

This model for calibration error shows a significant effect for errors on the 
order of one percent of value or greater, and shows the importance of accurate 
relative calibration. 

Next, an experiment was run to evaluate the interrelationships of two 
parameters. The effect of the IMC gain state on classification accuracy vs. 
meteorological range was studied. Figure 29. shows the result. It appears that 
the IMC gain state has a greater effect on improving classification accuracy in 
hazy, or low meteorological range, atmospheres. But, even in clear 
atmospheres the higher gain state shows an improvement. 
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Average Classification Accuracy (%) 



Meteorological Range (Km) 

Figure 29. Classification accuracy vs. meteorological range for several 

IMC gain states. 
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6. Relationships between SNR, Classification Accuracy, 
Separability, and Feature Selection 


In this section we discuss computing the SNR based on the combined 
features, and present results showing the relationships between this SNR, class 
separability and classification accuracy for several feature sets. 


In calculating the SNR of a feature, the signal and noise levels of the L 
component wavelength bands are combined. In the case of the voltage SNR, 
the signal levels are simply added together, as are the variances of the noise 
levels. This is shown in equation 6. 


Feature Voltage 



( 6 ) 


For the power SNR, the signal levels cannot be simply added because of 
the band-to-band correlation present in the reflectance data. Here, the signal 
variance is the sum of the individual variances, plus terms due to the covariance 
between each pair of bands I and m, combined in the feature as in equation (7). 


Feature Power SNR = 10 log 10 


c 

i 


L M 


° m G| Gm 


1 


1-1 m-1 


o 2 . ,+c 2 . ,+c 2 

shot.l read, I quant 




'J 


( 7 ) 


An experiment was then carried out to determine if the SNR of a 
particular feature had any effect on its importance in classification. The 
combined voltage and power SNR’s of the feature set shown in Table 9 were 
computed and the results shown in Figure 30. 
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■ Voltage SNR 
H Power SNR 


Feature Number 

Figure 30. SNR’s for features of Table 9. 


These features were then ranked according to the measures of Tab e . 
Then the effect of these rankings were measured by computing accuracy vs. 
number of features ordered by each ranking measure. The results are shown i 

Figure 31 . 


Measure 

Rankina order 

Bandwidth 
Voltage SNR 
Power SNR 
V-P SNR 

Decreasing bandwidth of feature 
Decreasing Voltage SNR of feature 
Decreasing Power SNR of feature 
Decreasing difference between Voltage 
and Power SNR of the feature 


Table 11. Various rankings of the feature set. The ranking order is for 
increasing feature number. 


29 


4 / 20/89 




100 



>* 

o 

<0 

3 

o 

o 

< 


c 

o 

♦* 

(0 

o 


to 

to 

« 

o 



Number of Features Used in Classification 


Figure 31 . Classification accuracy vs. number of features for several 
different orderings of the feature set. 

The ranking by bandwidth, as in Table 9, appears to have the best 
performance for small numbers of features. For one feature, the feature with the 
highest power SNR performed the best, and above 9 or 10 features, all rankings 
performed similarly. 

An observation of these results is that the SNR of any individual feature is 
a poor predictor of that feature’s importance when included in a multi-feature 
classification algorithm. 

To obtain an overall measure of the SNR for a set of features, the method 
behind equations (6) and (7) was extended to combining the signals and noise 
from all wavelength bands contributing to the feature set. 

To relate these combined SNR's to classification accuracy, a scatter plot 
was made of the SNR’s for the various configurations used to produce Figures 
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22 through 27, and the resultant classification accuracies. This is shown in 
Figure 32 for the voltage and power feature SNR's. 



Average Classification Accuracy (%) 

Figure 32. Classification accuracy vs. combined feature SNR. 

Attempts to fit a linear or polynomial equation to these plots yielded a 
correlation coefficient of 0.71 for the best fit, that being a 3rd order polynomial. 
While there is a general relationship here between SNR and classification 
accuracy, they appear to be loosely correlated at best. 

Also in this experiment, a multiclass implementation of the transformed 
Bhattacharyya separability measure was used to compute the effect of the 
parameters on separability. Equation (8) shows the two class transformed 
Bhattacharyya distance. 


B. = 1 - e' a 

ij 


where, 


£. + 1 . 

_j [ 

2 


-i 

(M. - M.) ♦ 1 log. 


(£. + 1 )/2 ' 


( 8 ) 


Here, Mj is the feature mean vector of class i, while Ej is the feature 
covariance matrix of class i. The multiclass separability is computed using the 
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apriori probability weighted sum over all classes i, j of the two class distances, 
as in equation (9). 


Separability = 


^ 2 ^ p( “ )p( “i )B ii 
i-i j-i 


( 9 ) 


The plot of the classification accuracy vs. separability is shown in Figure 
33. The p 2 value of the linear best fit line is 0.93. Thus, over this range of 
classification accuracy and separability there is a high correlation between the 
two. 
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Separability 

Figure 33. Classification accuracy vs. transformed Bhattacharyya distance. 

A plot of the separability vs. SNR is shown in Figure 34. The best fit line 
has an p 2 = 0.62 for the voltage SNR and p 2 = 0.63 for the power SNR. 

A tentative conclusion from these results is that while separability and 
classification accuracy are highly correlated, much less correlation exists 
between either the voltage or the power SNR and classification 
accuracy/separability. 
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Figure 34. Transformed Bhattacharyya separability vs. combined feature SNR. 

Several sets of six features (shown in Table 12) were used to evaluate 
their classification performance. 


Feature 

SFD 

TM 

WSNR 

NSNR 

SSFD 

SSNR 

1 

0.42-0.66 

0.45-0.52 

0.40-0.70 

0.51-0.56 

0.59 

0.54 

2 

0.66-0.84 

0.52-0.60 

0.77-0.90 

0.81-0.86 

0.75 

0.84 

3 

0.70-0.92 

0.63-0.69 

1.00-1.10 

1.02-1.07 

0.81 

1.04 

4 

1 .48-1 .64 

0.76-0.90 

1.15-1.30 

1.20-1.25 

1.56 

1.11 

5 

1.98-2.20 

1.45-1.75 

1.50-1.74 

1.59-1.64 

2.10 

1.61 

6 

2.20-2.40 

2.08-2.35 

1.97-2.40 

2.16-2.21 

2.30 

2.19 


Table 12. Wavelength bands combined for the various feature 
sets. The various feature sets are defined as SFD = Spectral 
Feature Design algorithm, TM = Landsat Thematic Mapper, WSNR 
= Wide Signal-to-Noise Ratio, NSNR = Narrow Signal-to-Noise 
Ratio, SSFD = Single band Spectral Feature, SSNR = Single 
band Signal-to-Noise Ratio. 

The SNR features were chosen based upon regions of high SNR. These 
various sets were chosen to see how classification accuracy and combined 
signal-to-noise ratios compared. Figure 35 shows the combined SNR for the 
various feature sets, while Figure 36 shows the resultant classification accuracy 
for the baseline image used in this report. 
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Figure 35. SNR's for the various feature sets. 
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Figure 36. Classification accuracy for the various feature sets. 
Clearly, the SFD features performed the best for this data set. 
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The robustness of the spectral feature design algorithm was then studied 
by comparing the accuracy of the various feature sets in classifying scenes 
created from other data sets than those from which the features were derived. 
Two other scenes were tested. One was created from reflectance data of three 
varieties of spring wheat. The second was from an artificial data set created 
from the covariance matrices of barley, pasture, and grain sorghum, with the 
same mean vector (that of barley). This was done to decrease the class 
separability and increase any difference in the classification accuracy due to the 
differing feature sets. When the original means were used all feature sets gave 
99.9% accuracy. Tables 13 and 14 gives the specific fields from the LARS field 
data base combined for the two data sets. 


Classes 

Field 

Number of Observations 

Spring Wheat 

118 

13 

154 

29 


199 

28 


291 

28 


292 

16 

Total = 114 

SW 1809 

296 

28 


303 

58 

Total = 86 

SW Mix 

75 1 

. - 13 — 


281 

55 

Total = 68 


Table 13. Classes and fields used to compute statistics for the Spring Wheat 
test scene. The data is from Hand County, South Dakota, on July 26, 1978. 


i Classes 

Field 

Number of Observations 

Barley 

153 1 

26 

231 

21 


294 

55 

Total = 102 

Pasture 

109 

43 


194 

45 


198 

24 

Total = 112 

Grain Sorghum 

110 

15 

256 

51 


275 

36 

Total = 102 


Table 1 4. Classes and fields used to compute statistics for the Crops 3 
test scene. The data is from Hand County, South Dakota, on July 26, 1978. In 
simulating the scene, the mean vector of the Barley class was used for all three 

classes. 
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Figure 37 shows the resulting accuracy for the various features and for 
the two test scenes. The crops 3 scene was tested for IMC = 1 and 8 to see the 
effect of increasing the SNR of the instrument on classification performance. 
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Figure 37. Classification accuracy for the various feature sets over two test 

scenes. 

In all cases the features formed from the wavelengths used in the 
Landsat TM performed the best. The features derived from the high SNR 
regions also performed well. Compared to these two feature sets, the SFD 
feature set performed poorly. 

This leads to a tentative conclusion that over varying scenes, the 
features derived from the reflectance of a different crop type, or scene, perform 
less well at classification than features derived from signal-to-noise regions of 
the instrument, or even the wavelength bands used in the Thematic Mapper. 
This is not surprising since the SFD procedure is intended to be case-specific; it 
is intended to provide features optimal for its design case, as compared to being 
optimal in the general case. 


36 


4 / 20/89 



7. Summary and Conclusions 

This report summarizes the current status of research into predicting the 
performance of the HIRIS instrument under varying system parameters, signai- 
to-noise conditions, and feature extraction methods. 

Several tentative conclusions can be made. 

• The current noise levels of the HIRIS instrument are adequate for use 
in classification studies. 

• The radiometric resolution of 12 bits is well matched to the other 
sources of noise. At 8 bits, the quantization error becomes the 
dominant source of noise. 

• The SNR of a feature obtained by combining bands is a poor 
predictor of the effectiveness of that feature alone in multivariate 
classification algorithms. 

• The combined SNR of a feature set is a "loose" predictor of 
classification performance. 

• Classification and separability are highly correlated. 

• The features derived from the spectral feature design algorithm work 
well for a scene created from the statistics used in their design. 
However, they are not broadly optimum, and can be outperformed by 
more simply chosen feature sets. 

Further work is necessary before etching these conclusions in stone. 
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